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AMENDMENTS TO THE CLAIMS 

The following listing of claims will replace all prior versions and/or listings of claims in the 
application: 

1. (Cancelled) 

2. (Currently Amended) A computer-readable medium as in claim [[1]] 14, wherein said 
passback direction is defined based on at least one of an image of said video monitored area and 
video of said video monitored area. 

3. (Currently Amended) A computer-readable medium as in claim [[1]] 14, wherein said 
passback direction is user defined. 

4. (Original) A computer-readable medium as in claim 3, wherein said passback direction is 
determined via a graphical user interface to identify said passback direction in at least one of an 
image of said video monitored area and video of said video monitored area. 

5. (Original) A computer-readable medium as in claim 4, wherein said graphical user 
interface is adapted to permit a user to draw said passback direction on at least one of an image of 
said video monitored area and video of said video monitored area. 

6. (Currently Amended) A computer-readable medium as in claim [[1]] 14, wherein said 
defining said passback direction comprises learning said passback direction through observation of 
said video monitored area. 

7. (Original) A computer-readable medium as in claim 6, wherein said learning said 
passback direction comprises: 
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learning a normal direction for said video monitored area based on said observation of said 
video monitored area; and 

determining said passback direction based on said normal direction. 

8. (Original) A computer-readable medium as in claim 6, wherein said passback direction is 
learned as a function of time. 

9. (Original) A computer-readable medium as in claim 8, wherein said passback direction is 
learned for a first time period, and wherein said defining said passback direction further comprises 
learning at least one additional passback direction for said video monitored area for at least one 
additional time period based on said observation of said video monitored area. 

10. (Currently Amended) A computer-readable medium as in claim [[[1]] 14, wherein said 
passback direction is defined in association with at least one of a region of interest of said video 
monitored area, a time interval of detection, and a sensitivity of detection. 

1 1 . (Currently Amended) A computer-readable medium as in claim [[1]] 14, wherein said 
accessing video comprises accessing video in real time from a video camera. 

12. (Currently Amended) A computer-readable medium as in claim [[1]] 14, wherein said 
accessing video comprises accessing stored video. 

13. (Cancelled) 

14. (Currently Amended) A computer-readable medium , comprising software encoded 
thereon to detect passback events, which software when executed by a computer system, causes the 
computer system to perform operations comprising a method of as in claim 13, wh e r e in said 
e xtracting tracks compris e s : 

defining a passback direction for a video monitored area; 
accessing video collected from the video monitored area; 
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extracting tracks from the collected video, comprising: 

extracting foreground from said collected video to obtain extracted 
foreground; 

detecting trackable features based on said extracted foreground; 

tracking said trackable features based on said extracted foreground to obtain 
extracted tracks; and 

filtering said extracted tracks; and 
wher e in said analyzing video furth e r compris e s detecting passback events based on the 
passback direction and said extracted tracks ; and 

initiating an action based on the detected passback events . 

15. (Original) A computer-readable medium as in claim 14, wherein said extracting 
foreground is based on pixel statistics of a current frame and at least one past frame. 

16. (Original) A computer-readable medium as in claim 14, wherein said extracting 
foreground is based on three-frame motion differencing. 

1 7. (Original) A computer-readable medium as in claim 14, wherein said detecting trackable 
features comprises: 

subdividing foreground of said collected video into cells; 
determining if each cell is appropriate for tracking; and 
designating appropriate cells for tracking as trackable features. 

18. (Currently Amended) A computer-readable medium as in claim 17, wherein said 
determining if each cell is appropriate for tracking is based on at least one of texture in each cell, 
intensity range in each cell, number of different intensity values in each cell, presence of at least one 
edge in each cell, and or presence of at least one corner in each cell. 

19. (Original) A computer-readable medium as in claim 14, wherein said tracking said 
trackable features comprises: 
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correlating each trackable feature with previous tracked features to obtain correlated trackable 
features and correlated previous tracked features; 

comparing directions of said correlated trackable features and said correlated previous* 
tracked features; 

designating said correlated trackable features as tracked features if said directions are 
consistent; and 

updating said correlated previous tracked features with said correlated trackable features if 
said directions are consistent; and 

wherein said filtering said extracted tracks comprises filtering said tracked features. 

20. (Original) A computer-readable medium as in claim 19, wherein said correlating each 
trackable feature comprises performing two-dimensional correlation in an area predicted from said 
previous tracked features to contain said trackable feature. 

21 . (Original) A computer-readable medium as in claim 19, wherein said correlating each 
trackable feature comprises performing one-dimensional correlation on a horizontal projection and a 
vertical projection of an area predicted from said previous tracked features to contain said trackable 
feature. 

22. (Currently Amended) A computer-readable medium as in claim 19, wherein said 
updating said correlated previous tracked features is based on at least one of a direction aad or an age 
of said correlated trackable features. 

23. (Currently Amended) A computer-readable medium as in claim 14, wherein said filtering 
said extracted tracks comprises at least one of filtering noise from said extracted tracks, filtering out 
nuisance extracted tracks from said extracted tracks, filtering out extracted tracks from said extracted 
tracks that are not sufficiently long lived, and or filtering out extracted tracks from said extracted 
tracks that are inconsistent with other nearby extracted tracks. 
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24. (Currently Amended) A computer-readable medium , comprising software encoded 
thereon to detect passback events, which software when executed by a computer system, causes the 
computer system to perform operations comprising a method of as in claim 13, wh e r e in said 
e xtracting tracks comprises : 

defining a passback direction for a video monitored area; 
accessing video collected from the video monitored area; 
extracting tracks from the collected video, comprising: 

extracting foreground from said collected video to obtain extracted 
foreground; 

determining optical flow based on said extracted foreground to obtain said 
extracted tracks; and 

filtering said extracted tracks; 
detecting passback events based on the passback direction and the extracted tracks; and 
initiating an action based on the detected passback events . 

25. (Original) A computer-readable medium as in claim 24, wherein said extracting 
foreground is based on pixel statistics of a current frame and at least one past frame. 

26. (Original) A computer-readable medium as in claim 24, wherein said extracting 
foreground is based on three-frame motion differencing. 

27. (Original) A computer-readable medium as in claim 24, wherein said determining optical 
flow comprises: 

determining current flow vectors for foreground pixels of said extracted foreground; 
comparing current and previous flow vectors to obtain consistent current flow vectors; and 
aggregating consistent current flow vectors into cumulative flow vectors. 

28. (Original) A computer-readable medium as in claim 24, wherein said filtering said 
extracted tracks comprises at least one of filtering noise from said extracted tracks, filtering out 
nuisance extracted tracks from said extracted tracks, filtering out extracted tracks from said extracted 
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tracks that are not sufficiently long lived, and filtering out extracted tracks from said extracted tracks 
that are inconsistent with other nearby extracted tracks. 

29. (Cancelled) 

30. (Currently Amended) A computer-readable medium , comprising software encoded 
thereon to detect passback events, which software when executed by a computer system, causes the 
computer system to perform operations comprising a method of a s in claim 29, wh e r e in said 
comparing said track and said passback dir e ction compris e s : 

defining a passback direction for a video monitored area; 
accessing video collected from the video monitored area; 

comparing a track determined from the collected video to the passback direction to detect 
whether any passback event occurred, comprising: 

representing said track as a track vector; 

representing said passback direction as a passback direction vector; 
determining a dot product of said track vector and said passback direction 
vector; [[and]] 

comparing said dot product to a threshold to determine whether said track 
vector is substantially aligned with said passback direction; and 

when said track vector is substantially aligned with said passback direction, 
filtering said track vector by at least one of spatial filtering aad or temporal filtering to 
determine whether a sufficient number of track vectors are substantially aligned with 
said passback direction; 

detecting passback events based on the determination; and 
initiating an action based on the detected passback events . 

31. (Cancelled) 



-7- 



Applicant: LIPTONetal. 
Appl.No. 10/766,949 



32. (Currently Amended) A computer-readable medium , comprising software encoded 
thereon to detect passback events, which software when executed by a computer system, causes the 
computer system to perform operations comprising a method of as in claim 3 1 : 

defining a passback direction for a video monitored area; 
accessing video collected from the video monitored area; 
analyzing the collected video; 

detecting passback events in the analyzed video based on the passback direction; and 
generating a report for each detected passback event , wherein said report comprises at least 
one of a time of each detected passback event, a date of each detected passback event, a direction of 
each detected passback event, a location of each detected passback event, a size of a crossing area of 
each detected passback event, an indication of occlusion of each detected passback event, at least one 
snapshot of each detected passback event, at least one video of each detected passback event, a 
number of detected passback events over a time interval, and or a time histogram of detected 
passback events over a time interval. 

33. (Currently Amended) A computer-readable medium as in claim [[1]] J4, wherein said 
initiating said action comprises initiating a triggered response for each detected passback event. 

34. (Currently Amended) A computer-readable medium as in claim 33, wherein said 
triggered response comprises at least one of initiating an alarm, controlling an audible alarm system, 
controlling a silent alarm system, accessing an alerting device, accessing an alerting system, sending 
an alert, logging alert data to a database, taking a snapshot of each detected passback event, culling a 
snapshot of each detected passback event from said video, recording video of each detected passback 
event, controlling a camera to zoom in on each detected passback event, controlling a camera to track 
each detected passback event, performing recognition of each detected passback event, closing at 
least one door, and or controlling an access control system to at least one of lock, unlock, open, and 
close at least one portal. 
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35. (Currently Amended) A computer-readable medium as in claim [[1]] 14, said m e thod 
further comprising defining at least one of an option for reporting any detected passback event aad or 
an option for responding to any detected passback event. 

36. (Currently Amended) A computer-readable medium as in claim [[1]] 14, wherein said 
method is adapted to detect passback events in real time. 

37. (Currently Amended) A method to detect passback events comprising: 
defining a passback direction for a video monitored area; 

accessing video collected from said video monitored area; 

extracting tracks from the collected video, comprising: 

extracting foreground from the collected video to obtain extracted foreground, 

detecting trackable features based on the extracted foreground, 

tracking the trackable features based on the extracted foreground to obtain 

extracted tracks, and 

filtering the extracted tracks; 

detecting analyzing said vid e o coll e ct e d from said vid e o monitor e d ar e a to d e t e ct any 
passback events in said video monitored area based on said passback direction and extracted tracks ; 
and 

initiating an action based on [[any]] the detected passback events. 

38. (Currently Amended) A system to detect passback events, comprising: 

at least one video camera to monitor at least one video monitored area and to obtain video of 
said video monitored area; 

an analysis system coupled to said video camera, said analysis system comprising a computer 
system and a computer-readable medium, said computer-readable medium comprising software to 
control said computer system according to a method, said method comprising! 

extracting foreground from the collected video to obtain extracted foreground, 
detecting trackable features based on the extracted foreground. 
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tracking the trackable features based on the extracted foreground to obtain 
extracted tracks, and 

filtering the extracted tracks to detect passback events in said video monitored 
area based on a passback direction and the extracted tracks for said video monitored 
area; and 

a user interface coupled to said analysis system. 

39. (Currently Amended) An apparatus to detect passback events adapted to perform 
operations comprising a method of: 

defining a passback direction for a video monitored area; 

accessing video collected from said video monitored area; 

extracting tracks from the collected video, comprising: 

extracting foreground from the collected video to obtain extracted foreground, 

detecting trackable features based on the extracted foreground, 

tracking the trackable features based on the extracted foreground to obtain 

extracted tracks, and 

filtering the extracted tracks; 

detecting analyzing said video coll e ct e d from said vid e o monitor e d ar e a to d e t e ct any 
passback events in said video monitored area based on said passback direction and extracted tracks : 
and 

initiating an action based on [[any]] die detected passback events. 

40. (Currently Amended) The apparatus of claim 39, wherein said apparatus comprises 
application-specific hardware to emulate at least one of a computer and/or or software adapted to 
perform said operations d e fining, said acc e ssing, said analyzing, and said initiating . 

41 . (New) A method to detect passback events, comprising: 
defining a passback direction for a video monitored area; 
accessing video collected from the video monitored area; 
extracting tracks from the collected video, comprising: 
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extracting foreground from the collected video to obtain extracted 
foreground, 

determining optical flow based on the extracted foreground to obtain the 
extracted tracks, and 

filtering the extracted tracks; 
detecting passback events based on the passback direction and the extracted tracks; and 
initiating an action based on the detected passback events. 

42. (New) A method to detect passback events, comprising: 
defining a passback direction for a video monitored area; 
accessing video collected from the video monitored area; 

comparing a track determined from the collected video to the passback direction to detect 
whether any passback event occurred, comprising: 

representing the track as a track vector, 

representing the passback direction as a passback direction vector, 
determining a dot product of the track vector and the passback direction 

vector, 

comparing the dot product to a threshold to determine whether the track 
vector is substantially aligned with the passback direction, and 

when the track vector is substantially aligned with the passback direction, 
filtering the track vector by at least one of spatial filtering or temporal filtering to 
determine whether a sufficient number of track vectors are substantially aligned with 
the passback direction; 

detecting passback events based on the determination; and 
initiating an action based on the detected passback events. 

43. (New) A method to detect passback events, comprising: 
defining a passback direction for a video monitored area; 
accessing video collected from the video monitored area; 
analyzing the collected video; 
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detecting passback events in the analyzed video based on the passback direction; and 
generating a report for each detected passback event, wherein the report comprises at least 
one of a time of each detected passback event, a date of each detected passback event, a direction of 
each detected passback event, a location of each detected passback event, a size of a crossing area of 
each detected passback event, an indication of occlusion of each detected passback event, at least one 
snapshot of each detected passback event, at least one video of each detected passback event, a 
number of detected passback events over a time interval, or a time histogram of detected passback 
events over a time interval. 

44. (New) A system to detect passback events, comprising: 

a video camera to monitor a video monitored area and obtain video of the video monitored 

area; 

an analysis system coupled to the video camera, the analysis system comprising a computer 
system and a computer-readable medium, the computer-readable medium comprising software to 
control the computer system according to a method, the method comprising: 

extracting foreground from the obtained video to obtain extracted foreground, 
determining optical flow based on the extracted foreground to obtain extracted 
tracks, and 

filtering the extracted tracks to detect passback events in the video monitored 
area based on a passback direction and the extracted tracks; 

and 

a user interface coupled to the analysis system. 

45. (New) A system to detect passback events, comprising: 

a video camera to monitor a video monitored area and obtain video of the video monitored 

area; 

an analysis system coupled to the video camera, the analysis system comprising a computer 
system and a computer-readable medium, the computer-readable medium comprising software to 
control the computer system according to a method, the method comprising: 
determining a track in the obtained video, 
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representing the track as a track vector, 

representing a passback direction as a passback direction vector, 
determining a dot product of the track vector and the passback direction 

vector, 

comparing the dot product to a threshold to determine whether the track 
vector is substantially aligned with the passback direction, 

when the track vector is substantially aligned with the passback direction, 
filtering the track vector by at least one of spatial filtering or temporal filtering to 
determine whether a substantial number of track vectors are substantially aligned with 
the passback direction, and 

detecting the passback events in the video monitored area based on the 
number of track vectors which are substantially aligned with the passback direction; 

and 

a user interface coupled to the analysis system. 

46. (New) A system to detect passback events, comprising: 

a video camera to monitor a video monitored area and obtain video of the video monitored 

area; 

an analysis system coupled to the video camera, the analysis system comprising a computer 
system and a computer-readable medium, the computer-readable medium comprising software to 
control the computer system according to a method, the method comprising: 

analyzing the obtained video to detect passback events in the video monitored 

area based on a passback direction for the video monitored area, and 
generating a report for each detected passback event, 
wherein the report comprises at least one of a time of each detected passback event, a date of each 
detected passback event, a direction of each detected passback event, a location of each detected 
passback event, a size of a crossing area of each detected passback event, an indication of occlusion 
of each detected passback event, at least one snapshot of each detected passback event, at least one 
video of each detected passback event, a number of detected passback events over a time interval, or 
a time histogram of detected passback events over a time interval; and 



-13- 



Applicant: LIPTONetal. 
Appl.No. 10/766,949 

a user interface coupled to the analysis system. 

47. (New) An apparatus to detect passback events adapted to perform operations comprising 
a method of: 

defining a passback direction for a video monitored area; 
accessing video collected from the video monitored area; 
extracting tracks from the collected video, comprising: 

extracting foreground from the collected video to obtain extracted 
foreground, 

determining optical flow based on the extracted foreground to obtain the 
extracted tracks, and 

filtering the extracted tracks; 
detecting passback events based on the passback direction and the extracted tracks; and 
initiating an action based on the detected passback events. 

48. (New) An apparatus to detect passback events adapted to perform operations comprising 
a method of: 

defining a passback direction for a video monitored area; 
accessing video collected from the video monitored area; 

comparing a track determined from the collected video to the passback direction to detect 
whether any passback event occurred, comprising: 

representing the track as a track vector, 

representing the passback direction as a passback direction vector, 
determining a dot product of the track vector and the passback direction 

vector, 

comparing the dot product to a threshold to determine whether the track 
vector is substantially aligned with the passback direction, and 

when the track vector is substantially aligned with the passback direction, 
filtering the track vector by at least one of spatial filtering or temporal filtering 
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to determine whether a sufficient number of track vectors are substantially aligned 
with the passback direction; 

detecting passback events based on the determination; and 
initiating an action based on the detected passback events. 

49. (New) An apparatus to detect passback events adapted to perform operations comprising 
a method of: 

defining a passback direction for a video monitored area; 
accessing video collected from the video monitored area; 
analyzing the collected video; 

detecting passback events in the analyzed video based on the passback direction; and 
generating a report for each detected passback event, wherein the report comprises at least 
one of a time of each detected passback event, a date of each detected passback event, a direction of 
each detected passback event, a location of each detected passback event, a size of a crossing area of 
each detected passback event, an indication of occlusion of each detected passback event, at least one 
snapshot of each detected passback event, at least one video of each detected passback event, a 
number of detected passback events over a time interval, or a time histogram of detected passback 
events over a time interval. 
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